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Results  in  Brief 


Mr.  Chairman  and  Members  of  the  Subcommittee: 

We  are  pleased  to  be  here  today  to  discuss  the  Department  of  Defense’s 
(DOD)  efforts  to  improve  its  program  for  relocating  militeuy  personnel  and 
their  families.  This  has  been  a  long,  complicated  process.  Since  1994,  DOD 
has  been  engaged  in  initiatives  to  reengineer  the  personal  property 
program  to  simplify  current  processes,  control  program  costs,  ensure 
quality  of  service  by  adopting  commercial  business  processes 
characteristic  of  world-class  businesses,  and  relieve  carriers  of 
DOD-unique  terms  and  conditions.  The  U.S.  Transportation  Command 
(USTRANSCOM)  is  tasked  with  evaluating  the  pilots  to  determine  which 
pilots,  or  portions  thereof,  could  provide  better  long-term  results  for  DOD! 
The  statement  of  managers  in  the  conference  report  on  the  DOD 
Appropriations  Act  for  1997  directed  us  to  validate  the  results  and  savings 
achieved  from  any  personal  property  program  before  DOD  proposes 
further  expansion  of  such  programs?  My  testimony  today  will  focus  on  our 
ongoing  work  related  to  the  Army’s  Hunter  pilot  and  DOD’s  plans  to 
evaluate  other  ongoing  and  planned  pilot  programs.  In  assessing  the 
Army’s  evaluation  of  the  Hunter  Pilot,  we  focused  on  (1)  the  evaluation 
methodology  and  (2)  the  Army  Audit  Agency’s  (AAA)  validation  of  the  test 
and  baseline  data. 


We  agree  that  DOD  needs  to  reengineer  its  personal  property  moving 
process  to  improve  the  poor  quality  of  service  military  members  are 
receiving.  We  support  the  use  of  pilots  as  a  means  to  test  new  concepts. 
Although  this  process  has  been  ongoing  since  the  early  1990s,  DOD  is  not 
yet  in  a  position  to  determine  what  changes  are  needed. 

We  were  unable  to  validate  the  reported  results  of  the  Army’s  evaluation  of 
the  Hunter  pilot  because  of  weaknesses  in  the  evaluation  methodology  and 
the  data.  However,  the  lessons  learned  from  the  pilot  do  provide 
information  that  should  be  useful  to  DOD  as  it  assesses  and  conducts  its 
pilot  efforts.  Further,  the  Hunter  pilot  provides  military  personnel  with 


*The  mission  of  USTRANSCOM,  which  is  DOD’s  single  manager  of  all  defense  transportation  services, 
is  to  provide  global  air,  land,  and  sea  transportation  to  meet  national  security  needs,  both  in  time  of 
peace  and  time  of  war.  USTRANSCOM  executes  its  mission  through  three  component  commands: 
Military  Traffic  Management  Command  (MTMC)  for  land  transportation  and  port  operations,  the 
Military  Sealift  Command  for  sea  transport,  and  the  Air  Mobility  Command  for  air  transport. 

^House  Report  104-863,  Sept.  28, 1996. 
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services  that  were  not  previously  available  during  the  moving  process, 
including  personal  move  counseling  and  coordination,  full  replacement 
value  for  lost  or  dcimaged  household  goods,  and  visibility  of  the  shipment 
throughout  the  move. 

Key  pilot  results  that  we  were  unable  to  validate  included  the  level  of 
customer  satisfaction  and  costs.  Although  customer  surveys  were 
conducted,  the  results  were  inconclusive  because  of  the  methods  used. 
While  the  Army  indicated  that  the  estimated  pilot  cost  was  higher  than  the 
baseline  cost,  we  could  not  validate  the  extent  to  which  the  pilot  cost 
exceeded  the  baseline  cost.  Further,  we  could  not  calculate  the  baseline 
and  pilot  costs  due  to  weaknesses  in  the  Army’s  methodology  and  data 
reliability.  As  for  the  participation  of  small  businesses,  we  confirmed  that 
33  percent  of  the  pilot  shipments  were  awarded  by  Cendant  Mobility  to 
small  business  carriers  and  agents. 

DOD  has  three  pilot  programs  underway  to  improve  its  current  personal 
property  program  and  is  proposing  a  fourth  pilot.  The  general  plan  is  to 
evaluate  the  results  of  these  pilots  and  use  that  information  to  develop  a 
redesigned  Department-wide  relocation  program.  To  achieve  this 
objective,  it  is  imperative  that  DOD  develop  a  well  thought  out  strategy 
with  clear  time  lines  for  testing  each  of  the  approaches,  and  more 
importantly,  an  evaluation  methodology  that  will  produce  credible  and 
accurate  information  to  be  used  in  making  a  final  decision.  However,  plans 
for  accomplishing  these  tasks  and  milestones  for  implementing  a  new 
process  have  not  yet  been  finalized. 


Background 


DOD  has  long  been  concerned  about  the  quality  service  members  receive 
from  its  nearly  $3  billion  annual  program  to  transport,  store,  and  manage 
the  household  goods  and  unaccompanied  baggage  of  its  personnel.  Past 
problems  include  poor  service  from  its  movers,  excessive  incidence  of  loss 
or  damage  to  service  members’  property,  and  high  claims  costs  to  the 
government.  All  of  these  problems  contribute  to  a  poor  quality  of  service 
for  persons  using  the  system. 


MTMC’s  Reengineering  DOD  first  proposed  reengineering  its  personal  property  program  5  years 

Efforts  Began  in  1 994  Deputy  Commander  in  Chief,  USTRANSCOM, 

directed  MTMC,  the  Army  component  of  USTRANSCOM  and  program 
manager  for  DOD’s  Personal  Property  Shipment  and  Storage  Program,  to 
reengineer  the  personal  property  program.  On  June  15, 1995,  the  House 
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MTMC  Pilot  Commenced  in  . 
1999 


Committee  on  National  Security,  concurring  that  DOD  must  pursue  a 
higher  level  of  service,  directed  that  DOD  undertake  a  pilot  program  to 
implement  commercial  business  practices  and  standards  of  service? 
MTMC  planned  to  award  contracts  for  the  new  program  pursuant  to  the 
competitive  acquisition  system  provisions  (chapter  137  of  title  10  of  the 
U.S.  Code  and  the  primary  implementing  regulation  contained  in  the 
Federal  Acquisition  Regulation) .  Expressing  congressional  concerns  about 
the  impact  that  the  competition  system  and  any  nonstandard  commercial 
business  practice  requirements  might  have  on  small  businesses,  the 
statement  of  managers  accompanying  the  1997  DOD  Authorization  Act 
directed  that  DOD  report  on  the  impact  of  the  pilot  program  on  small 
business.  After  reviewing  the  defense  reports  on  small  business  impact, 
the  House  Committee  on  National  Security  was  still  concerned  that 
MTMC’s  pilot  program  did  not  satisfactorily  address  issues  raised  by  the 
small  moving  companies  and  directed  that  the  Secretary  of  Defense 
establish  a  working  group  of  military  and  industry  representatives  to 
develop  an  alternative  pilot  program. 


Although  the  working  group  reached  a  consensus  on  many  issues, 
including  a  set  of  program  goals,  it  could  not  reach  agreement  on  the 
approach  to  take  for  the  pilot  test.  Consequently,  the  two  sides  presented 
separate  proposals.  In  November  1996,  we  reported  that  in  our  assessment 
MTMC’s  proposal  met  the  goals  of  reengineering  the  personal  property 
program  to  a  greater  extent  than  the  industry  plan.^ 

The  MTMC  pilot  program  implementation,  delayed  by  numerous  bid 
protests,  finally  commenced  in  January  1999.®  The  pilot  reengineers  the 
existing  program  for  50  percent  of  the  moves  originating  in  three  states: 
North  Carolina,  South  Carolina,  and  Florida.  The  pilot  will  run 
concurrently  with  the  existing  MTMC-managed  program  at  installations  in 
the  three  states  and  will  involve  approximately  18,500  emnual  moves.  The 


^The  House  Committee  on  National  Security  is  now  called  the  House  Armed  Services  Committee. 

^See  Defense  Transportation:  Reengineering  the  POP  Personal  Property  Program(GAO/NSIAD-97-49. 
Nov.  27, 1996)  for  more  detailed  information  on  the  MTMC  pilot  effort. 

^Solicitation  disputes  in  the  following  decisions:  Aalco  Forwarding.  Inc.,  et  airB-277241.8.  B-277241.9, 
Oct.  21, 1 997') :  Aalco  Forwarding.  Inc,  et  al  (B-277241.12,  B-277241.13,  Dec.  29.19971:  Aalco  Forwarding 
Inc,  et  al.  (B-277241.15,  Mar.  11, 1998):  Aalco  Forwarding.  Inc,  et  alfB-277241.16.  Mar.  11. 1998):  Aalco 
Forwarding.  Inc.,  et  al  (B-277241,  277241.14,  June  8, 1998^,  and  Aalco  Forwarding.  Inc.. 
et  al.  (B-277241.  B-277241. 20.  B-277241. 21.  July  1. 1998). 
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key  features  of  the  pilot  program  include  selecting  carriers  based  on 
service  member  satisfaction  and  past  performance  rather  than  simply 
price:  achieving  stronger  carrier  commitment  with  long-term  contracts;  and 
offering  full  replacement  value  protection  and  direct  claims  settlement  to 
users.  MTMC’s  pilot  will  run  for  a  3-year  test  period  (1  year  with  two  1-year 
option  periods),  which  will  end  in  September  2002. 


Hunter  Pilot  Effort  Began  in  Separately  from  the  MTMC  pilot  program,  the  Army  decided  to  determine 
jgg0  whether  the  commercial  business  practice  of  outsourcing  could  alleviate 

known  problem  areas.  The  pilot  was  initiated  in  February  1996  as  a 
quality-of-life  effort  to  improve  the  relocation  process  and  to  test 
commercial  business  practices  in  a  military  environment  at  Hunter  Army 
Airfield,  Savannah,  Georgia.  On  January  31, 1997,  the  Army  selected  PHH 
Relocation,  a  relocation/move  management  company  (today  named 
Cendant  Mobility,  located  in  Bethesda,  Maryland),  as  the  contractor  for  the 
pilot.  In  July  1997,  Cendant  Mobility  began  relocation  operations  at  Hunter 
Army  Airfield,  consisting  of  point-to-point  move  management,  personal 
move  counseling  and  coordination,  assistance  in  buying/selling  a  residence, 
24-hour  in-transit  visibility  of  the  shipment,  direct  claims  settlement  with 
the  service  member,  full  replacement  value,  and  a  single  point  of  contact 
for  the  member,  that  were  not  previously  available  under  the  existing 
system.  The  contract  will  end  on  September  30, 1999. 


The  Navy  initiated  a  separate  pilot  effort  in  1997  to  test  the  option  of 
allowing  service  members  to  select  their  carrier,  giving  them  more  control 
over  the  relocation  process  to  meet  their  specific  needs.  The  pilot,  which 
commenced  in  January  1998,  is  not  intended  to  replace  the  existing 
MTMC-managed  program  and  affects  a  very  small  number  of  military 
personnel.  Presently,  the  option  is  offered  for  shipments  originating  in  the 
Puget  Sound,  Washington;  San  Diego,  California:  Norfolk,  Virginia;  and 
Groton,  Connecticut  areas.  Participation  is  limited  to  Navy  military 
members  with  permanent  change  of  station  orders  from  the  participating 
sites.  Eligible  participants  must  limit  shipments  to  at  least  3,000  pounds 
and  to  costs  between  $2,500  and  $25,000  (and  it  excludes  shipments  from 
nontemporary  storage  or  warehouse) .  The  service  member  can  select  a 
mover  from  a  list  of  carriers  that  are  self-certified  as  small  businesses  and 
approved  by  MTMC.  As  of  March  1999, 169  Navy  service  members  had 
selected  this  option.  This  pilot  does  not  have  a  specific  end  date. 


Navy  Pilot  Effort  Began  in 
1997 
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DOD  Pilot  Effort  Proposed  On  February  12, 1999,  DOD  stated  that  it  intended  to  be^n  a  fourth  test, 
in  1999  called  the  Full  Service  Moving  Project,  with  certain  modifications  based  on 

lessons  learned  from  the  Army  Hunter  test.  The  test  will  involve  a  larger 
volume  of  moves  and  will  include  the  National  Capital  Region  (17,000 
annual  moves),  Georgia  (26,000  annual  moves),  and  North  Dakota  (2,000 
annual  moves) .  Like  the  Army  pilot,  it  is  intended  to  replace  the  existing 
MTMC-managed  program  by  using  a  contractor  or  contractors  to  provide 
both  transportation  and  move  management  services.  Presently,  there  is  no 
official  start  date  for  this  project. 


USTRANSCOM  to  Evaluate 
All  Pilot  Programs 


To  oversee  all  personal  property  pilots,  the  Deputy  Under  Secretary  of 
Defense  (Logistics)  has  tasked  USTRANSCOM  to  ensure  consistency  in 
evaluation  criteria  and  assessment.  USTRANSCOM  plans  to  evciluate  the 
pilots  to  determine  which  pilots  or  tests,  or  portions  thereof,  could  provide 
better  long-term  results.  It  will  then  recommend  the  follow-on  course  of 
action  and  time  lines  for  implementation  throughout  DOD.  Presently, 
USTRANSCOM  is  working  with  the  services  to  develop  a  common  set  of 
data  measures  to  evaluate  the  pilot  projects’  results. 


Results  of  Army  Hunter 
Pilot  Are  Inconclusive 


Because  of  the  weaknesses  in  the  Army’s  evaluation  methodology  and  data, 
we  were  unable  to  validate  all  reported  results  of  the  Hunter  pilot  program. 
Moreover,  the  Army’s  evaluation  plan  methodology  was  not  an  effective 
tool  for  collecting  and  analyzing  the  pilot  results.  However,  the  lessons 
learned  from  the  pilot  program  do  provide  useful  information  to  DOD  as  it 
conducts  and  assesses  its  pilot  efforts.  Also,  the  pilot  program  is  providing 
services  emd  benefits  that  were  not  previously  available,  including 
point-to-point  move  management,  personal  move  counseling  and 
coordination,  direct  claims  settlement,  assistance  in  buying/selling  a 
residence,  and  visibility  of  the  shipment  throughout  the  move. 


The  Army’s  Assessment  and 
Evaluation  Approach 


Through  the  first  12  months  of  operations,  the  Army  reported  that  the 
Hunter  pilot  was  successful  in  demonstrating  that  commercial  practices 
can  be  applied  to  the  military  relocation  process.  That  is.  Army  officials 
stated  that  they  were  able  to  contract  with  a  relocation  company  to  provide 
services  similar  to  those  available  in  the  private  sector.  To  evaluate  the 
pilot,  in  January  1997,  the  Army  began  developing  a  test  evaluation  plan 
that  defined  roles  and  responsibilities,  test  factors,  and  the  process  for 
capturing  data  related  to  each  test  factor,  and  stated  how  test  data  would 
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be  compared  to  that  from  the  existing  program.  The  plan  stated  that  AAA 
would  be  responsible  for  validating  the  baseline  and  test  data  and  for 
providing  the  Army  with  a  report  on  the  results  of  the  test.  AAA  conducted 
a  review  and  its  findings  cire  incorporated  in  our  evaluation.  We  reported  in 
June  1998  that  the  Army  needed  to  further  define  the  various  factors  and 
measurements.®  The  Army  provided  more  clarity  and  a  revised 
methodology  in  the  evaluation  plan  dated  August  12, 1998.  This 
represented  the  third  such  revision,  which  was  completed  more  than  a  year 
after  the  pilot  had  begun.  The  modified  plan  indicated  that  the  pilot  results 
would  be  reviewed  on  the  basis  of  three  factors:  quality  of  life,  cost,  and 
impact  on  small  business — and  would  use  a  5-point  scoring  system. 

Quality  of  life  was  the  key  element  the  Army  used  in  its  evaluation, 
consisting  of  1  point  each  for  its  three  sub-factors — customer  satisfaction, 
average  claims  settlement  time,  and  percentage  of  direct  deliveries — along 
with  1  point  each  for  the  other  two  factors — total  cost  and  impact  on  small 
business,  for  a  total  of  5  points. 

In  its  October  1998  evaluation  report  and  February  and  March  1999 
supplements,  the  Army  reported  to  us  that  the  pilot  project  earned  a 
passing  score  on  each  of  the  test  factors  that  it  measured  over  the  12-month 
period  ending  June  30, 1998.  The  Army  determined  that  the  Hunter  pilot 
was  a  success  because  the  factor  points  exceeded  the  minimum  threshold 
of  3.0  based  on  a  5.0-point  scale.  The  Hunter  pilot  test  earned  3.75  points, 
as  follows: 

•  customer  satisfaction  improved  by  over  1 1  percent  (0.25  point) , 

•  the  initial  offer  to  settle  claims  averaged  9  days  (1  point) , 

•  100  percent  of  eligible  shipments  were  delivered  directly  to  the  service 
member  without  storage  in  transit  (1  point), 

•  Pilot  project  costs  exceeded  baseline  costs  by  18.6  percent  (0.50  point) , 
and 

•  33  percent  of  the  shipments  were  awarded  to  small  businesses  (1  point) . 

The  Army  stated  that  it  also  considered  other  factors,  generally  related  to 
transportation  process  improvements,  and  did  not  rely  on  test  scores  alone 
to  determine  the  pilot  a  success.  These  benefits  included  simplifying  the 
price  structure  and  claims  process,  providing  pre-audit  and  certification 
services,  and  minimizing  government-unique  requirements.  Each  service 


^Defense  Transportation:  The  Army's  Hunter  Pilot  Project  to  OutsourcR  Relocation  Services 

(GAO/NSIAD-98-149.  June  10, 1998). 
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Evaluation  Weaknesses  Led 
to  Inconclusive  Results 


Inconclusive  Results  for 
Customer  Satisfaction 


member  worked  with  one  personal  move  coordinator  who  was 
accountable  for  direct  delivery  at  destination,  timely  service,  and  prompt 
resolution  of  any  problems.  This  is  in  contrast  to  the  current  process 
where  a  service  member  may  need  to  contact  four  to  five  different  offices 
over  the  course  of  a  single  move.  Neither  AAA  nor  we  reviewed  these 
benefits  cited  by  the  Army  because  they  were  not  included  as  measurable 
factors  in  the  evaluation  plan. 


We  identified  a  number  of  shortfalls  with  the  Army’s  evaluation 
methodology  and  data.  In  several  instances,  the  data  provided  to  support 
the  measurements  did  not  demonstrate  achievement  of  the  reported 
results.  Most  importantly,  we  could  not  confirm  that  customer  satisfaction 
improved  by  11.5  percent.  Further,  we  could  not  confirm  that  all  eligible 
shipments  were  delivered  directly  without  incurring  in-transit  storage 
costs,  that  offer  of  claims  settlement  time  averaged  9  days,  and  that  the 
pilot  program  cost  18.6  percent  more  than  the  baseline  cost.  We  reviewed 
the  methodology  and  data  for  each  factor — qucdity  of  life  (consisting  of 
three  sub-factors:  customer  satisfaction,  claims  settlement  time,  and  direct 
deliveries);  costs;  emd  impact  on  small  business. 

Because  service  members  were  surveyed  multiple  times  and  the  survey 
methodology  was  flawed,  the  Army’s  reported  results  on  customer 
satisfaction  were  compromised.  Therefore,  we  could  not  validate  that 
customer  satisfaction  improved  by  1 1.5  percent.  However,  data  gathered  in 
these  surveys  did  provide  indicators  of  customer  satisfaction  as  well  as 
provide  lessons  learned  for  DOD,  USTRANSCOM,  and  the  Army  that  may 
be  useful  for  conducting  their  pilot  efforts. 

To  measure  customer  satisfaction,  the  Army  said  it  would  use  its  contractor 
(Battelle’s  Pacific  Northwest  National  Laboratory)  to  survey  members  at 
Himter  and  those  making  similar  moves  at  Fort  Stewart,  Georgia. 
Responses  from  service  members  at  both  locations  would  be  compared. 
The  Army  told  us  that  Battelle  would  conduct  the  survey,  but  instead,  the 
Army  used  USTRANSCOM’s  survey,  conducted  by  Electronic  Data  Systems 
(EDS),  to  evaluate  and  score  the  pilot  project.  Initially,  the  Army  informed 
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US  that  it  did  not  use  Battelle’s  results  because  the  draft  report  was  difficult 
to  interpret  and  inconclusive.  It  subsequently  expressed  concern  over  a 
disclaimer  on  the  survey  results.^ 

Nonetheless,  service  members  were  surveyed  multiple  times,  possibly  by 
four  different  entities.  EDS  and  Battelle  (the  survey  contractors),  Cendant 
Mobility  (which  was  required  by  its  contract  to  survey  a  sample  of 
customers),  and  the  carrier  (for  its  own  quality  control)  each  may  have 
contacted  the  same  service  members  to  query  them  on  their  move 
experience.  As  a  result,  the  customer  satisfaction  results  were 
compromised  and  produced  inconclusive  results.  EDS  reported  that  some 
service  members  refused  to  respond  to  the  survey  and  that  it  does  not 
know  how  many  of  those  who  did  respond  only  gave  perfunctory  answers. 
In  a  caveat  in  its  report,  Battelle  stated  that  it  was  aware  that  some 
individuals  were  interviewed  multiple  times. 

In  addition,  the  contractors’  survey  results  varied  significantly.  While  EDS 
reported  that  11.5  percent  of  respondents  were  more  satisfied  with  then- 
current  relocation  experience  (the  pilot),  Battelle  reported  that  only 
3  percent  of  its  respondents  were  more  satisfied?  We  are  uncertain  of  the 
specific  reasons  for  these  differences.  However,  we  do  know  that  the 
question  cisked  by  the  contractors  to  determine  customer  satisfaction  was 
not  identical,  and  that  the  number  of  respondents  in  the  two  surveys  varied 
significantly.  This  may  have  contributed  to  the  differences.  EDS  had  a 
much  lower  response  rate,  53  percent,  them  Battelle,  which  achieved  an 
89-percent  response  from  service  members  on  which  their  results  were 
calculated.  Further,  EDS  used  a  75-percent  significance  level,  while 
Battelle  calculated  responses  using  a  95-percent  significance  level.  The 
latter  is  a  standard  used  in  social  science  research? 


^The  Army  said  it  subsequently  learned  that  the  Department  of  Energy  requires  such  a  disclaimer  in  all 
studies  performed  for  government  entities  and  that  the  disclaimer  had  no  relevance  to  the  results 
reported  by  Battelle. 

^Battelle  also  reported  that  the  3-percent  figure  was  “statistically  insignificant,”  meaning  that  the 
variance  in  responses  from  the  two  groups  (personnel  at  Hunter  and  at  Fort  Stewart)  was  not  great 
enough  to  conclude  that  their  move  experiences  differed  significantly, 

^Significance  level  is  the  likelihood  that  a  true  population  value  may  be  rejected  by  a  statistical  test.  In 
the  case  of  the  two  surveys,  for  example,  Battelle  used  a  95-percent  and  EDS  used  a  75-percent 
probability  that  the  differences  in  relocation  satisfaction  levels  were  due  to  something  other  than 
random  occurrence.  Battelle  found  no  significant  difference  in  satisfaction  levels,  with  a  5-percent 
chance  of  reaching  an  incorrect  conclusion.  EDS  did  find  a  significant  difference  but  with  a  25-percent 
chance  of  reaching  an  incorrect  conclusion. 
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Limited  Data  on  Claims 
Settlement  Time 


Other  data  obtained  by  the  surveys,  but  not  measured  by  the  Army,  provide 
indicators  of  customer  satisfaction  and  highlights  areas  that  could  be 
addressed  in  future  evaluations.  For  example,  the  Battelle  survey  found 
that  61  percent  of  the  Hunter  respondents  were  more  satisfied  with  their 
most  recent  (pilot)  moving  experience  than  with  their  prior  move. 
Eighty-seven  percent  of  these  respondents  were  satisfied  or  veiy  satisfied 
with  the  carrier’s  responsiveness.  Both  surveys  also  indicated  that  service 
members  relocating  within  the  continental  United  States  were  generally 
more  satisfied  with  the  pilot  process  than  personnel  moving  overseas  were. 
The  Army  cited  several  examples  of  problems  with  overseas  moves  that 
indicated  greater  dissatisfaction  by  those  service  members,  and  stated  that 
it  has  taken  action  to  correct  some  of  the  problems. 

The  surveys  also  showed  that  service  members  liked  some  unique  features 
the  pilot  offered,  such  as  one-on-one  counseling  and  the  simplified  claims 
process.  However,  few  service  members  took  advantage  of  many  of  the 
additional  services  that  Cendant  Mobility  offered,  particularly  assistance  in 
house  hunting.  Moreover,  officials  representing  The  Military  Coalition, 
which  represents  five  million  active  duty,  retired,  reserve,  and  National 
Guard  personnel  and  their  families,  told  us  that  more  could  be  done  to 
tailor  the  range  of  services  offered  by  the  relocation  company  to  meet 
unique  military  needs.  Such  services,  they  said,  might  include  more 
assistance  in  finding  rental  property  because  military  personnel  tend  to 
rent  rather  than  purchase,  and  providing  assistance  for  spouses  seeking 
employment. 

Another  lesson  learned  related  to  access  to  survey  data  by  outside 
reviewers.  AAA  and  we  were  unable  to  verify  the  survey  data  because  the 
independent  survey  contractors,  citing  privacy  act  restrictions,  refused  to 
grant  access  to  the  original  survey  documentation.  The  Army  stated  that,  in 
future  pilot  projects,  it  would  ensure  that  service  members  are  not 
subjected  to  multiple  surveys  and  would  establish  a  better  survey  method 
for  determining  customer  satisfaction. 

Due  to  weaknesses  in  the  Army’s  data  collection  and  aneilysis,  we  could  not 
validate  that  the  initial  offer  to  settle  claims  occurred  on  average  in 
9  days  after  a  completed  claim  form  was  received  by  Cendant  Mobility. 
While  the  claims  settlement  time  is  one  approach  to  measuring  the  claims 
process,  it  does  not  capture  the  entire  claims  process  and  does  not 
measure  the  extent  the  process  represented  an  improvement  over  the 
existing  system.  The  pilot  process,  however,  does  offer  benefits  to  the 
service  member  in  terms  of  full  replacement  value  protection.  For 
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Direct  Delivery  Results  May  Be 
Overstated 


example,  household  effects  are  insured  for  their  replacement  value  (up  to 
$75,000)  rather  than  their  depreciated  value  (which  had  been  capped  at 
$40,000).  The  pilot  process  also  requires  less  work  for  the  service  member 
since  the  contractor  provides  estimates  on  lost  and  damaged  household 
goods,  and  the  contractor  pays  the  service  member  directly. 

The  claims  settlement  time  measurement  is  based  on  the  contract 
requirement  that  the  contractor  offer  settlement  to  a  member  within 
30  days  after  receiving  a  claim  for  damaged  or  lost  property.  Consequently, 
the  Army  developed  a  threshold  for  claims  settlement  of  30  days.  Under 
the  current  process,  we  were  told  that  it  takes  an  average  30  days  for  the 
U.S.  Army  Claims  Service  to  review  a  completed  claim  form  and  offer 
settlement,  and  it  takes  the  Defense  Finance  and  Accounting  Service  an 
additional  day  to  process  the  payment.  Service  members  may  appeal  to  the 
contractor  for  reconsideration  by  providing  additional  information  or  file 
an  appeal  with  the  Army  if  resolution  with  the  contractor  could  not  be 
achieved — a  process  that  can  take  more  than  2  years  to  complete.  While 
we  found  that  some  claims  remained  unpaid  and  unsettled  because  the 
claims  were  put  “on  hold”  by  the  service  member,  the  Army  reported  that 
no  one  under  the  Hunter  pilot  filed  an  appeal  as  of  January  1999. 

The  Army  reported  that  Cendant  Mobility  took  an  average  9  days  to  offer 
settlement  for  lost  or  damaged  property,  but  AAA  and  we  could  not  confirm 
this  because  the  Army  had  not  collected  the  necessary  data.  AAA 
determined,  and  we  concur,  that  Cendant  Mobility  may  have  taken  as  long 
as  an  average  28  days  to  offer  settlement.  This  figure  is  based  on 
measuring  the  time  between  the  date  the  claim  was  signed  and  the  date  the 
member  was  offered  settlement.  The  difference  between  the  two  estimates 
is  primarily  due  to  an  average  19-day  period  between  the  date  the  claim  was 
signed  and  the  date  the  contractor  reported  that  it  entered  the  claim  into  its 
database.  Although  Cendant  Mobility  officials  stated  that  its  standard 
company  practice  is  to  enter  a  claim  into  the  database  the  same  day  a  claim 
is  received,  delays  could  be  caused  by  (1)  the  form  being  mailed  some  time 
after  it  is  dated,  (2)  mail  delays,  and  (3)  contractor  delays  in  entering  the 
claim  into  the  database.  Army  officials  stated  that,  due  to  the  difficulties  in 
measuring  claims  settlement  time,  they  would  change  their  data  collection 
and  analysis  of  this  sub-factor  in  future  pilot  programs. 

We  could  not  confirm  that  all  eligible  direct  deliveries  were  made  without 
in-transit  storage  because  the  necessary  data  was  not  collected.  Direct 
delivery  is  defined  by  the  Army  as  delivery  of  a  shipment  to  a  service 
member’s  residence  without  storage  in  transit.  A  shipment  is  eligible  for 
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direct  delivery  only  if  it  has  a  destination  address  before  the  shipment  is 
offered  for  delivery.  Direct  delivery  is  a  contract  performance 
measurement,  and  the  contractor  is  paid  an  incentive  for  maintaining  a 
direct  delivery  rate  of  over  60  percent.  Consequently,  the  Army  threshold 
for  this  measurement  is  60  percent.  Measuring  direct  delivery  in  this 
manner  does  not  provide  the  Army  sufficient  information  to  determine 
whether  the  results  represent  an  improvement  over  the  current  system. 

Neither  AAA  nor  we  could  confirm  that  the  shipments  that  were  directly 
delivered  to  service  members  were  the  only  ones  eligible  for  direct  delivery 
because  the  necessary  information  on  all  shipments  was  not  collected. 
That  is,  other  shipments  may  have  been  candidates  (eligible)  under  the 
Army  criteria;  however,  AAA  and  we  could  not  determine  which  shipments 
had  addresses  prior  to  being  offered  for  delivery  (and  were  thus  eligible) 
but  ended  up  requiring  in-transit  storage.  The  Army  acknowledged  the 
difficulty  in  validating  eligible  direct  deliveries  and  stated  that  in  the  future, 
it  would  consider  using  in-transit  storage  costs  as  a  test  factor.  In  the 
February  5, 1999,  supplemental  information  provided  to  us,  the  Army 
stated  that  74  percent  of  Army-wide  shipments  required  in-transit  storage, 
compared  to  66  percent  requiring  temporary  storage  during  the  Army  pilot. 

Service  members  benefit  from  having  household  goods  delivered  directly 
to  their  homes  because  the  practice  limits  additional  handling  of  their 
property,  reducing  opportunities  for  loss  and  damage.  Also,  the 
government  avoids  temporary,  in-transit  storage,  which  is  costly  and  hard 
for  the  Army  to  control.  Historically,  household  goods  shipments 
frequently  require  temporary  storage  because  service  members  often  do 
not  know  at  the  time  of  shipment  their  new  address  and/or  service 
members  will  not  immediately  move  to  the  new  duty  station  due  to 
vacations  or  military  requirements. 

Cost  Data  Were  Inaccurate  and  We  could  not  validate  that  18.6  percent  represented  the  additional  cost  of 

Incomplete  the  pilot  project  over  the  baseline  cost  due  to  weaknesses  in  the  Army’s 

methodology  and  the  reliability  of  overhead  cost  data.  As  we  reported  in 
June  1998,  developing  overhead  costs  historically  has  been  difficult  in  the 
government,  including  DOD,  because  such  data  are  often  unreliable  and 
unavailable  (see  app.  I  for  a  detailed  breakdown  of  pilot  and  baseline 
costs) . 

Neither  AAA  nor  we  could  validate  some  overhead  costs.  Specifically,  we 
could  not  confirm  the  accuracy  of  the  overhead  costs  attributed  to  MTMC 
because  the  Army  used  data  from  fiscal  year  1994.  MTMC  has  changed 
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significantly  since  that  time  due  to  downsizing,  but  AAA  could  not  obtain 
updated  costs  from  MTMC  representing  the  current  organization  to 
determine  the  reasonableness  of  its  overhead  cost.  AAA  and  we  also  agree 
that  the  costs  associated  with  redueing  the  cost  of  processing 
documentation  (claims,  invoices,  and  inbound)  would  not  result  in  a  cost 
reduction  to  the  government  unless  the  activities  that  perform  these 
functions,  such  as  the  Defense  Finance  and  Accounting  Service,  take 
corresponding  action  to  reduce  their  costs  (for  example,  eliminating 
personnel  to  reflect  the  drop  in  the  workload) . 

AAA  and  we  also  identified  other  costs  that  were  not  included  in  the  Army’s 
analysis,  such  as  the  costs  related  to  awarding  the  Hunter  contract  and  the 
contractor’s  use  of  foreign-flag  vessels.  The  Army  could  not  provide  an 
estimated  dollar  value  associated  with  the  cost  of  developing  the 
solicitation,  reviewing  offers,  and  resolving  bid  protests.  While  we 
recognize  that  some  of  these  estimated  costs  may  be  infrequent  or  one-time 
costs,  they  should  be  considered.  Also,  the  Army  did  not  factor  the 
contractor’s  use  of  foreign-flag  vessels  into  the  pilot  project’s 
transportation  costs.^*^  While  individually,  costs  such  as  these  are  probably 
of  low  dollar  value  in  relation  to  other  costs  in  the  analysis:  collectively, 
they  could  have  an  impact  on  the  difference  between  the  baseline  and  pilot 
costs. 

According  to  the  Army,  some  of  the  reasons  for  the  higher  estimated  cost  of 
the  pilot  project  can  be  attributed  to  (1)  the  difficulty  in  calculating 
accurate  bciseline  costs,  which  the  Army  believes  are  understated:  (2)  the 
low  volume  of  moves — 1,400 — which  did  not  provide  enough  leverage  to 
negotiate  better  rates  and  discounts:  (3)  higher-than-expected  cost  of 
overseas  shipments:  (4)  relatively  high  imaccompanied  baggage  shipment 
rates,  which  could  have  been  lower  using  negotiated  rates:  (5)  the  packing 
allowance  for  “do-it-yourself’  moves  was  incorrectly  calculated  using 
commercial  rates  during  part  of  the  12-month  test  period,  which  resulted  in 
a  higher  rate:  and  (6)  the  fact  that,  generally,  quality  moving  services  cost 
more.  In  regard  to  the  high  cost  of  overseas  shipments,  the  Army  provided 
us  with  additional  details  of  pilot  project  costs  that  showed  a  44-percent 
increase  for  overseas  household  goods  shipments  and  a  2-percent  increase 
for  domestic  shipments  over  the  baseline  cost. 


*®Foreign-flag  vessels  are  those  ships  registered  in  foreign  countries. 
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Impact  on  Small  Business  Both  AAA  and  we  validated  that  Cendant  Mobility  awarded  33  percent  of 

the  shipments  to  small  businesses — 10  percent  to  small  business  carriers 
and  23  percent  to  small  business  agents.  The  Army  measured  the  impact  of 
the  pilot  on  two  types  of  small  business  providers  (carriers  and  agents)  and 
established  a  threshold  of  23  percent  to  demonstrate  successful 
participation  by  small  businesses.*^  The  Army  based  this  factor  on  section 
15(g)  of  the  Small  Business  Act,  which  establishes  a  governmentwide  goal 
for  participation  by  small  business  concerns  at  not  less  than  23  percent  of 
the  value  of  all  prime  contracts.  Of  the  790  billed  Hunter  shipments  during 
the  12-month  pilot,  the  Army  reported  that  261  (33  percent)  were  awarded 
by  Cendant  Mobility  to  small  businesses. 


Status  of  Plans  to 
Evaluate  the  Pilot 
Programs 


USTRANSCOM  is  still  in  the  process  of  finalizing  the  evaluation  plan  for 
other  ongoing  and  planned  relocation  pilot  programs.  It  is  proposing  to 
evaluate  the  pilots  on  the  same  three  factors  that  the  Army  used  in  its 
evaluation,  except  the  factors  wilt  be  defined  differently  and  will  use  a 
more  expansive  point  scale.  Unresolved  issues,  as  of  February  1999, 
include  the  development  of  a  method  that  recognizes  the  unique 
characteristics  and/or  process  improvements  of  each  pilot  program,  and 
the  validation  of  the  baseline  indirect  costs  that  will  be  used  for  each  of  the 
pilot  programs.  The  evaluation  plan  does  not  currently  include  an 
evaluation  of  the  Hunter  pilot,  but  has  used  lessons  learned  in  the  Hunter 
pilot  to  help  develop  the  plan.  We  provided  comments  to  USTRANSCOM  in 
this  process,  and  among  other  things,  encouraged  them  to  seek  expert 
methodological  advice  before  finalizing  the  evaluation  plan  to  enhance  the 
quality  of  USTRANSCOM’s  assessment. 


Summary 


Improving  DOD’s  personal  property  program  has  been  a  slow,  complex 
process.  DOD  and  the  services  have  spent  a  large  amount  of  time  and 
effort  to  dramatically  change  the  quality  of  service  their  military  customers 
receive.  We  support  these  efforts.  However,  before  any  final  conclusions 
can  be  reached,  DOD  must  have  accurate  and  credible  data  to  determine 
the  type  and  extent  of  changes  that  should  be  made.  To  facilitate  a  timely 
completion  of  this  process,  DOD  needs  to  (1)  develop  a  comprehensive 
strategy  for  testing  a  finite  number  of  approaches,  (2)  specify  time  lines  for 


"in  the  motor  freight  and  transportation  industry,  firms  with  annual  gross  revenues  of  $18.5  million  or 
less  are  classified  as  small  businesses.  See  Federal  Acquisition  Regulation  19.102. 
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implementation  cmd  completion,  and  (3)  ensure  that  it  has  a 
methodologically  sound  evaluation  plan  to  assess  each  pilot’s  attributes  in 
a  comparable  manner.  As  it  has  been  in  the  past,  it  is  important  for  DOD  to 
continue  taking  into  consideration  the  views  of  the  moving  and  relocation 
companies  that  will  be  affected  by  changes  to  the  current  program. 

Mr.  Chairman,  this  concludes  my  prepared  statement.  I  would  be  pleased 
to  answer  any  questions  you  or  members  of  the  Subcommittee  might  have 
at  this  time. 
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Appendix  I _ _ _ 

Pilot  and  Baseline  Costs  Calculated  by  the 
Department  of  the  Army  (Office  of  the  Deputy 
Chief  of  Staff  for  Logistics)  and  Validated  hy  Army 
Audit  Agency 


Cost  elements 

Estimated 
pilot  costs 

Estimated 
baseline  costs 

Transportation  costs 

1.  Transportation 

$3,005,229 

$2,102,384 

2.  Accessorials 

25,767 

30,655 

3.  Storage 

210,706 

211,655 

Total  transportation  costs 

$3,241,702 

$2,344,694 

Claims  paid  by  Army 

4.  Claims  paid  by  Army 

0 

$66,268 

Overhead  costs 


5.  Personnel 

$79,802 

$221,721 

6.  Management  price 

132,700 

0 

7.  MTMC  headquarters® 

0 

63,510 

8.  Automation 

0 

52,612 

9.  Voucher  processing 

4,614 

90,658 

10.  Inbound  processing 

0 

46,834 

1 1 .  Claims  processing 

0 

11,510 

12.  Pay  for  performance 

14,756 

0 

13.  Building  overhead 

28,321 

43,286 

14.  Telephone  and  copier 

1,281 

10,106 

15.  Consumables 

3,760 

5,189 

Total  overhead  costs 

$265,234 

$545,426 

Total  cost 

$3,506,936 

$2,956,388 

Pilot  costs 

Baseline 

costs 

Difference 

Percent  of 
difference 

Percent  pilot  costs  exceeded 
baseline  costs 

$3,506,936 

$2,956,388 

$550,548 

18.6% 

^Military  Traffic  Management  Command  (MTMC) 

Source:  Office  of  the  Deputy  Chief  of  Staff  for  Logistics,  Department  of  the  Army 
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Related  GAO  Products 


Defense  Transportation:  The  Army’s  Hunter  Pilot  Project  to  Outsource 
Relocation  Services  (GAO/NSIAD-98-149,  June  10, 1998). 

Defense  Outsourcing:  Better  Data  Needed  to  Support  Overhead  Rates  for 
A-7fi  Studies  (GAO/NSIAD-98-62.  Feb.  27, 1998). 

Defense  Transportation:  Reengineering  the  POD  Personal  Property 
Program  (GAO/NSIAD-97-49,  Nov.  27, 1996). 

Household  Goods:  Administrative  Changes  Would  Improve  POP’s 
Do-It-Yourself  Moving  Program  (GAO/NSIAD-94-226.  Sept.  27, 1994). 

POD  Commercial  Transportation:  Savings  Possible  Through  Better  Audit 
and  Negotiation  of  Rates  (GAO/NSIAD-92-61,  Dec.  27, 1991). 

Household  Goods:  Competition  Among  Commercial  Movers  Serving  POD 
Can  Be  Improved  (GAO/NSIAD-90-50,  Feb.  12, 1990). 
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Ordering  Information 

The  first  copy  of  each  GAO  report  and  testimony  is  free. 

Additional  copies  are  $2  each.  Orders  should  be  sent  to  the 
following  address,  accompanied  by  a  check  or  money  order  made 
out  to  the  Superintendent  of  Documents,  when  necessary,  VISA  and 
MasterCard  credit  cards  are  accepted,  also. 

Orders  for  100  or  more  copies  to  be  mailed  to  a  single  address  are 
discounted  25  percent. 

Orders  by  mail: 

U.S.  General  Accounting  Office 
P.O.  Box  37050 
Washington,  DC  20013 

or  visit: 

Room  1100 

700  4th  St.  NW  (corner  of  4th  and  G  Sts.  NW) 

U.S.  General  Accounting  Office 
Washington,  DC 

Orders  may  also  be  placed  by  calling  (202)  512-6000 

or  by  using  fax  number  (202)  512-6061,  or  TDD  (202)  512-2537. 

Each  day,  GAO  issues  a  list  of  newly  available  reports  and 
testimony.  To  receive  facsimile  copies  of  the  daily  list  or  any  list 
from  the  past  30  days,  please  call  (202)  512-6000  using  a  touchtone 
phone.  A  recorded  menu  will  provide  information  on  how  to  obtain 
these  lists. 

For  information  on  how  to  access  GAO  reports  on  the  INTERNET, 
send  an  e-mail  message  with  “info”  in  the  body  to: 

info@www.gao.gov 

or  visit  GAO’s  World  Wide  Web  Home  Page  at: 
http://www.gao.gov 
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